Image processing system, image processing device, and image processing method

ABSTRACT

An image processing system, an image processing device, and image processing method are provided. Each of the image processing system, the image processing device, and the image processing method includes dividing an image into a plurality of areas according to a predetermined division condition, measuring an optical transmittance of each of the plurality of divided areas or data correlating with the optical transmittance as transmittance data, calculating the optical transmittance or the data correlating with the optical transmittance from the image as the transmittance data, determining a parameter for adjusting a contrast for each of the plurality of areas according to the transmittance data obtained by the measuring or the calculating, and adjusting a contrast of each one of the plurality of areas using the parameter determined by the determining.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is based on and claims priority pursuant to 35U.S.C. §119(a) to Japanese Patent Application Nos. 2014-265512 and2015-146691, filed on Dec. 26, 2014, and Jul. 24, 2015, respectively, inthe Japan Patent Office, the entire disclosure of which is herebyincorporated by reference herein.

BACKGROUND

Technical Field

Embodiments of the present invention relate to an image processingsystem, an image processing device, and an image processing method.

Background Art

When an object outside or a remote object is captured or observed, forexample, by a vehicle-installed camera, a surveillance camera, andelectronic binoculars, the transmittance may deteriorate due to thelight that is dispersed, for example, by fog, mist, bai, and aparticulate matter (PM) 2.5. As a result, the image of the object has alow contrast (the degree of difference between the darker and lighterparts), and the viewability deteriorates.

Conventionally, the technology to divide an image into segmentsaccording to the texture and flatten the histogram of the divided areasfor the purpose of improving the contrast is known. This technology isreferred to as the contrast limited adaptive histogram Equalization(CLAHE). When a histogram is flattened using the CLAHE, a constraintvalue is imposed on the contrast improvement.

SUMMARY

Embodiments of the present invention described herein provide an imageprocessing system, an image processing device, and image processingmethod. Each of the image processing system, the image processingdevice, and the image processing method includes dividing an image intoa plurality of areas according to a predetermined division condition,measuring an optical transmittance of each of the plurality of dividedareas or data correlating with the optical transmittance astransmittance data; calculating the optical transmittance or the datacorrelating with the optical transmittance from the image as thetransmittance data, determining a parameter for adjusting a contrast foreach of the plurality of areas according to the transmittance dataobtained by the measuring or the calculating, and adjusting a contrastof each one of the plurality of areas using the parameter determined bythe determining.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of exemplary embodiments and the manyattendant advantages thereof will be readily obtained as the samebecomes better understood by reference to the following detaileddescription when considered in connection with the accompanyingdrawings.

FIG. 1 is a schematic diagram illustrating an example of the structureof an imaging device according to an embodiment of the presentinvention.

FIG. 2 is a functional block diagram illustrating an example of an imageprocessing device according to an embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating the contrast adjustment of animage, according to an embodiment of the present invention.

FIG. 4A, FIG. 4B, and FIG. 4C are diagrams illustrating the clippingcomputation performed in the contrast adjustment of an image, accordingto an embodiment of the present invention.

FIGS. 5A to 5F are diagrams illustrating an effect achieved by setting aclipping value, according to an embodiment of the present invention.

FIG. 6A and FIG. 6B are diagrams illustrating a method of calculating acontrast adjustment curve, according to the present embodiment.

FIG. 7A, FIG. 7B, and FIG. 7C are diagrams illustrating the gain used incolor saturation adjustment according to an embodiment of the presentinvention.

FIG. 8 is a flowchart of a flow of the contrast adjustment performed byan imaging device provided with the image processing device illustratedin FIG. 2.

FIG. 9A and FIG. 9B are diagrams illustrating a wide rim and a tele-rimaccording to an embodiment of the present invention.

FIG. 10 is a flowchart of another flow of the contrast adjustmentperformed by an imaging device, according to an embodiment of thepresent invention.

FIG. 11 is a functional block diagram illustrating another example of animage processing device according to an embodiment of the presentinvention.

FIG. 12 is a flowchart of the flow of the contrast adjustment performedby an imaging device provided with the image processing deviceillustrated in FIG. 11.

FIG. 13A and FIG. 13B are diagrams illustrating the detection of a roadarea, according to an embodiment of the present invention.

FIG. 14 is a functional block diagram illustrating further anotherexample of an image processing device according to an embodiment of thepresent invention.

FIG. 15 is a flowchart of the flow of the contrast adjustment performedby an imaging device provided with the image processing deviceillustrated in FIG. 14.

FIG. 16A and FIG. 16B are diagrams illustrating an example of areadivision according to an embodiment of the present invention.

FIG. 17 is a diagram illustrating an example of an area dividing methodaccording to an embodiment of the present invention.

FIG. 18 is a diagram illustrating a method of generating a dimmed image,according to an embodiment of the present invention.

FIG. 19A, FIG. 19B, and FIG. 19C are diagrams illustrating anothermethod of dividing the pixel values of a dimmed image, according to anembodiment of the present invention.

FIG. 20A, FIG. 20B, and FIG. 20C each illustrates an example of contrastadjustment according to an embodiment of the present invention.

FIG. 21 is a functional block diagram illustrating further anotherexample of an image processing system according to an embodiment of thepresent invention.

FIG. 22 is a flowchart of the flow of the contrast adjustment performedby an imaging device provided with the image processing deviceillustrated in FIG. 21.

FIG. 23 is a diagram illustrating another method of dividing the pixelvalues of a dimmed image, according to an embodiment of the presentinvention.

FIG. 24 is a diagram illustrating another method of dividing the pixelvalues of a dimmed image into two or more ranges, according to anembodiment of the present invention.

FIG. 25 is a flowchart of the processes of dividing the pixel values ofa dimmed image illustrated in FIG. 24.

FIG. 26 is a flowchart of the flow of the contrast adjustment includingthe processes of dividing the pixel values of a dimmed image illustratedin FIG. 25.

The accompanying drawings are intended to depict exemplary embodimentsof the present disclosure and should not be interpreted to limit thescope thereof. The accompanying drawings are not to be considered asdrawn to scale unless explicitly noted.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the presentinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“includes” and/or “including”, when used in this specification, specifythe presence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

In describing example embodiments shown in the drawings, specificterminology is employed for the sake of clarity. However, the presentdisclosure is not intended to be limited to the specific terminology soselected and it is to be understood that each specific element includesall technical equivalents that have the same structure, operate in asimilar manner, and achieve a similar result.

In the following description, illustrative embodiments will be describedwith reference to acts and symbolic representations of operations (e.g.,in the form of flowcharts) that may be implemented as program modules orfunctional processes including routines, programs, objects, components,data structures, etc., that perform particular tasks or implementparticular abstract data types and may be implemented using existinghardware at existing network elements or control nodes. Such existinghardware may include one or more central processing units (CPUs),digital signal processors (DSPs),application-specific-integrated-circuits (ASICs), field programmablegate arrays (FPGAs), computers or the like. These terms in general maybe collectively referred to as processors.

Unless specifically stated otherwise, or as is apparent from thediscussion, terms such as “processing” or “computing” or “calculating”or “determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical, electronicquantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

FIG. 1 illustrates an example of the configuration of an imaging deviceaccording to an embodiment of the present invention. The imaging devicemay be any of a digital camera, a camcorder, a pair of digitalbinoculars, a laptop personal computer (PC), a tablet PC, a smartphone,a mobile phone, a personal digital assistant (PDA), or the like. Notethat the camcorder includes, for example, a vehicle-installed camera anda surveillance camera. An imaging device 10 includes a lens unit 11, acapturing unit 12, a controller 13, an image memory 14, an operationunit 15, an output interface (I/F) 16, a transmittance measuring sensor17, and an image processing circuit 18.

The imaging device 10 is arranged such that the lens unit 11 faces anobject to be captured, and a shutter button that is a part of theoperation unit 15 is depressed when the object is captured. When theshutter button is depressed, the imaging device 10 lets the lightreflected at the object enter the capturing unit 12 through the lensunit 11. The lens unit 11 includes a plurality of lenses, a lens stop,and a focus adjustment mechanism. The focus adjustment mechanismprovides an autofocus (AF) function by which the focus is automaticallyadjusted.

The capturing unit 12 includes an imaging element such as acomplementary metal oxide semiconductor (CMOS) image sensor and acharge-coupled device (CCD) image sensor. The imaging element convertsthe light that has entered the imaging element into a voltage accordingto the intensity of the light. The capturing unit 12 includes an analogfront end (AFE) circuit that increases the gain of the voltage that isconverted by the imaging element. Moreover, the capturing unit 12includes an analog-to-digital (A/D) converter that converts the voltageoutput from the AFE circuit, i.e., an analog signal, into digital data.The digital data that is output from the A/D converter is output to thecontroller 13.

The controller 13 controls the imaging device 10 in its entirety. Thecontroller 13 receives the image data of the captured image, which isdigital data, from the capturing unit 12. Then, the controller 13 sendsthe received image data to the image processing circuit 18 such that theimage processing circuit 18 performs image processing on the sent imagedata. Moreover, the controller 13 stores the received image data in theimage memory 14, or outputs the received image data to an output devicethrough the output I/F 16. Further, the controller 13 controls the lensunit 11 according to the AF function provided for the lens unit 11, andcontrols the lens unit 11 according to an instruction received by theoperation unit 15 for, for example, changing the focal length of thelens, to switch the function or focal length of the lens unit 11.

The controller 13 includes a memory in which a program for performingthe control as described above or setting data such as set values usedby the program are stored, and a CPU that executes the program. Thememory can store the lens data of the lens such as the focal length ofthe lens in order to realize the AF function and change the focal lengthas described above.

The operation unit 15 includes an operation panel or various kinds ofbuttons such as a shutter button. The operation unit 15 obtains thedetail of the operation made to the imaging device 10, and transfers theobtained detail of the operation to the controller 13. For example, whenthe shutter button is depressed, the operation unit 15 notifies thecontroller 13 that the shutter button has been depressed. In this case,the controller 13 instructs the capturing unit 12 to capture an object.In addition to the detail of the operation, the operation unit 15 alsoreceives data indicating whether or not to adjust the fog, and transfersthe received data to the controller 13. Note that the adjustment of thefog is one of the contrast adjustment.

The output I/F 16 receives the image data stored in the image memory 14through the controller 13, and outputs the received image data to theoutput device. The output device may be any device that outputs imagedata, including, for example, a printer, a facsimile, a photocopier, anelectronic whiteboard, a projector, a PC, a tablet PC, and a smartphone.The output I/F 16 may be connected to the output device through a wiredconnection such as a high-definition multimedia interface (HDMI) cable(registered trademark), or through the network. Note that the networkmay be any known network such as the local area network (LAN), the widearea network (WAN), and the Internet, and the network may be through awired connection or through a wireless connection.

The transmittance measuring sensor 17 is sensor that measures theoptical transmittance, and may be used as a measuring unit. As thetransmittance may be measured on an image, the transmittance measuringsensor 17 is not necessarily installed in the imaging device 10. Thetransmittance measuring sensor 17 can measure the transmittance, forexample, by projecting infrared light or the like to measure its backscattering. A distance-measuring sensor may be used as the transmittancemeasuring sensor 17 to measure the distance between the imaging device10 and an object. In such as case, the transmittance is measured fromthe measured distance together with the image data. Alternatively, alaser beam having a specific wavelength may be emitted to calculate thetransmittance from the reflectance of the emitted laser beam. Aplurality of imaging devices may be used to capture multiple images withvarying angles to calculate the distance between the imaging device 10and the object from parallaxes of the captured images. In this case, thetransmittance is calculated from the calculated distance. Thetransmittance measuring sensor 17 is also controlled by the controller13. By using the transmittance measuring sensor 17 to measure thetransmittance, transmittance can be obtained with higher accuracy thancases in which the transmittance is obtained from an image.

The image processing circuit 18 receives the image data, which is outputfrom the capturing unit 12 and is stored in the image memory 14, throughthe controller 13, and performs image processing on the received imagedata to meet an output form that the output device demands. The imageprocessing circuit 18 also adjusts the contrast. The image processingcircuit 18 may be configured as a device separate from the controller 13as illustrated in FIG. 1. Alternatively, the image processing circuit 18may be configured as a program, and the controller 13 or the likecircuit or device may execute the program. A configuration in which thecontroller 13 or the like executes the program to perform theabove-described image processing may be provided as an image processingsystem instead of an image processing device.

FIG. 2 is a functional block diagram of the image processing circuit 18that serves as an image processing device, according to the presentembodiment. FIG. 2 illustrates an example configuration in which thetransmittance measuring sensor 17 illustrated in FIG. 1 is not used. Theimage processing circuit 18 includes a controller (CTL) I/F 20, an imagesignal processing (ISP) unit 21, a transmittance calculator 22 thatserves as a measuring unit, and a contrast adjuster 23. Note that whenthe transmittance measuring sensor 17 is used, the transmittancecalculator 22 is not necessary. When the transmittance measuring sensor17 is used in place of the transmittance calculator 22, thetransmittance measuring sensor 17 measures the transmittance and obtainsthe transmittance data, and the transmittance measuring sensor 17provides the contrast adjuster 23 with the obtained transmittance data.

The image data that is captured and output by the capturing unit 12illustrated in FIG. 1 is stored in the image memory 14 via thecontroller 13. The image data that is stored in the image memory 14 istransferred to the image processing circuit 18 through the controller13. The CTL I/F 20 is connected to the controller 13, and receives theimage data stored in the image memory 14 through the controller 13.Moreover, the CTL I/F 20 passes the image data on which image processinghas been performed to the controller 13.

The ISP unit 21 performs the image processing of cameras known in theart. Such image processing includes, for example, black-levelcorrection, shading correction, gain control, gamma correction, RGB(red, green, blue) conversion, filtering, and color correction. The ISPunit 21 outputs the image data on which image processing has beenperformed to the transmittance calculator 22 and the contrast adjuster23.

The transmittance calculator 22 performs image processing on the imagedata received from the ISP unit 21, and calculates and obtains thetransmittance. The method of calculation of the transmittance will bedescribed later in detail. The transmittance calculator 22 outputs thecalculated transmittance as transmittance data to the contrast adjuster23.

The contrast adjuster 23 uses the transmittance data received from thetransmittance calculator 22 to perform image processing. Morespecifically, the contrast adjuster 23 performs the CLAHE for thebrightness components, and performs the color saturation adjustment forthe color-tone components. For this reason, the contrast adjuster 23includes a CLAHE processor 24 that performs the CLAHE, and a colorsaturation adjuster 25 that performs color saturation adjustment. Afterthe image processing is complete, the contrast adjuster 23 stores theimage data on which the image processing has been performed in the imagememory 14 via the CTL I/F 20. The output device requests the image datafrom the controller 13 through the output I/F 16, and the controller 13outputs the image data stored in the image memory 14 to the outputdevice through the output I/F 16.

The contrast adjuster 23 does not only receive the image data on whichthe ISP unit 21 has performed image processing, but also receives theimage data directly from the CTL I/F 20. Moreover, the contrast adjuster23 can adjust the contrast of the image data that is directly receivedfrom the CTL I/F 20. The contrast adjuster 23 may perform any one of theCLAHE and color saturation adjustment instead of performing both theCLAHE and the color saturation adjustment.

Here, the CLAHE that the CLAHE processor 24 performs is brieflydescribed. In the CLAHE, as illustrated in FIG. 3, a captured image isdivided into a plurality of rectangular regions (tiles), and thehistogram is flattened for each of the tiles. By so doing, the contrastis enhanced. FIG. 3 illustrates an image captured in a foggy place. Itis to be noted that the density of the fog is even in that place, andthe optical transmittance changes according to the distance to theimaging device 10. In the following description, cases of fog will bedescribed. However, no limitation is intended thereby, and the fog maybe changed, for example, to a haze, bai, and particulate matter (PM)2.5.

Among the tiles that are equally divided into four in both vertical andhorizontal directions as illustrated in FIG. 3, the opticaltransmittance is low in the two upper tiles as a mountain at longdistance is captured therein, and the optical transmittance is high inthe bottom-left tile as a car at short distance is captured therein.Moreover, the optical transmittance has a middle value in thebottom-right tile as an object at middle distance is captured therein.

In a fog image that is captured in the fog, a tile with low opticaltransmittance, i.e., a tile where the distance to an object to becaptured is long and the fog is thickly captured, has low contrast, andthus the object to be captured is not clear. For this reason, theconstraint value of the contrast is made small, and the contrast is wellenhanced. On the other hand, a tile with high optical transmittance,i.e., a tile where the distance to an object to be captured is close andthe fog is thinly captured, is assigned with a large constraint value ofcontrast such that the contrast is not very much enhanced. A tile withintermediate optical transmittance is assigned with an intermediateconstraint value such that the contrast is moderately enhanced.Accordingly, an image where the object to be captured is not clear isadjusted in such a manner that the object to be captured becomes clearlyvisible, and an image where the object to be captured is clear iscontrolled in such a manner that the object to be captured is notexcessively adjusted. By so doing, the noise can be minimized.

When the thickness of the fog is even, as described above, theconstraint value of the contrast is changed according to the distance tothe object to be captured, and the contrast is thereby enhanced. Whenthe thickness of the fog is not even, the optical transmittance does notdepend on the distance. Accordingly, the constraint value of thecontrast is changed according to the measured transmittance or thecalculated transmittance, and the contrast is thereby enhanced.

The number of the tiles into which an image is divided by the CLAHE isnot limited to four as illustrated in FIG. 3, and may be divided into alarger number of tiles. For example, an image may be divided into nine,sixteen, and twenty-five tiles. The flattening of a histogram using theCLAHE may be implemented by adopting any known method. For example, themethod disclosed in JP-2006-195651-A may be adopted. As the contrastadjuster 23 adjusts an image into multiple tiles as described above, thecontrast adjuster 23 does not only serve as an adjuster that adjusts thecontrast, but also serves as a divider that divides an image into aplurality of areas.

The flattening of a histogram strongly enhances the contrast. For thisreason, a parameter called clipping value is introduced as a constraintvalue of the contrast in order to weaken the contrast enhancement, and aclipping computation of the histogram is performed.

The clipping computation is a computation used in the local histogramequalization. The local histogram equalization is a contrast enhancementtechnique known in the art, and contrast enhancement is performed inview of the local data of the image in the contrast enhancement. Theclipping computation is described with reference to FIG. 4A, FIG. 4B,and FIG. 4C. FIG. 4A indicates an area (tile) in an input image, andFIG. 4B is a distribution map of the distribution of the brightnessvalues of the pixels included in the tile illustrated in FIG. 4A.

The tile is given a prescribed clipping value, and when the number ofpixels exceeds the clipping value at some of the brightness values, theexceeding number of pixels as indicated in FIG. 4B by the oblique linesare removed. The sum of the number of the exceeding pixels iscalculated, and the obtained sum is divided by the total number of thebrightness values. Accordingly, the number of the pixels to bedistributed to each of the brightness values is calculated. Then, thenumber of the pixels to be distributed to each of the brightness valuesis added to the number of the pixels at each of the brightness valuesfrom which the number of the pixels exceeding the clipping value hasbeen removed. Accordingly, as illustrated in FIG. 4C, new numbers ofpixels at each of the brightness values are calculated. For example,when the sum of the number of pixels indicated by the oblique linesillustrated in FIG. 4B is 512 and the total number of the brightnessvalues is 256 due to 256-level gray scale, two pixels are distributedand added to each of all the brightness values due to the calculation“512/256=2”. In the clipping computation, the distribution of thenumbers of pixels at the brightness values as calculated above isredefined.

FIGS. 5A to 5F are diagrams illustrating an effect achieved by setting aclipping value, according to the present embodiment. In the distributionmaps where the number of pixels exceeding a clipping value has not yetbeen removed as illustrated in FIG. 5A and FIG. 5D on the left side, thebrightness values of a certain divided tile distribute between α and β.In FIG. 5A, FIG. 5B, and FIG. 5C, the number of pixels exceeding aclipping value k₁ is removed. In a similar manner, in FIG. 5D, FIG. 5E,and FIG. 5F, the number of pixels exceeding a clipping value k₂ (k₂<k₁)is removed. The sum of the number of the exceeding pixels is calculated,and the obtained sum is divided by the total number of the brightnessvalues. Accordingly, the number of the pixels to be distributed to eachof the brightness values is calculated. The obtained number of pixels isadded to each of all the brightness values. As a result, thepost-clipping distribution maps as illustrated in FIG. 5B and FIG. 5E inthe center are obtained. The post-clipping distribution maps arereferred to, and the numbers of pixels at the brightness values between0 to 255 are added and accumulated, and are normalized. As a result, theconversion curves that indicate the converted brightness values at thebrightness values are obtained as illustrated in FIG. 5C and FIG. 5F theright side. In the present embodiment, the normalization is done inorder to bring the values into the range of 0 to 255.

Such a conversion curve may be calculated by adding up the numbers ofpixels at the brightness values on all such occasions. Alternatively, alook-up table (LUT) that is obtained by adding up the numbers of pixelsat the brightness values is stored in a memory or the like in advance,and a conversion curve may be selected from such a LUT.

FIG. 6A and FIG. 6B are diagrams illustrating a method of calculating acontrast adjustment curve, according to the present embodiment. Morespecifically, a method of calculating a conversion curve from apost-clipping distribution map will be described in detail withreference to FIG. 6A and FIG. 6B. FIG. 6A illustrates a post-clippingdistribution map, and FIG. 6B illustrates a conversion curve obtainedfrom the distribution map illustrated in FIG. 6A. As illustrated in FIG.6A, the number of pixels is constant in the range where the brightnessvalue is 0 to α. In the range where the brightness value is 0 to α, apost-conversion brightness value is obtained by adding and accumulatinga fixed number of pixels to a pre-conversion brightness value.Accordingly, as illustrated in FIG. 6B, the conversion curve in therange where the brightness value is 0 to α has a fixed degree ofinclination. In a similar manner, in the range where the brightnessvalue is γ to δ and the range where the brightness value is β to 255, apost-conversion brightness value is obtained by adding and accumulatinga fixed number of pixels to a pre-conversion brightness value.Accordingly, as illustrated in FIG. 6B, the conversion curve in therange where the brightness value is γ to δ and the range where thebrightness value is β to 255 has a fixed degree of inclination.

Note that the number of pixels to be added is greater in the range wherethe brightness value is γ to δ than in the range where the brightnessvalue is 0 to α and the range where the brightness value is β to 255.Accordingly, the inclination in the range where the brightness value isγ to δ is steeper than the inclination in the range where the brightnessvalue is 0 to α and the range where the brightness value is β to 255. Inthe range where the brightness value is α to γ, as illustrated in FIG.6A, the number of pixels is not constant, and there is a tendency ofsharp increase. Accordingly, the conversion curve in the range where thebrightness value is α to γ, which is obtained by adding and accumulatinga number of pixels, becomes, as illustrated in FIG. 6B, a sharplyincreasing curve instead of a straight line. By contrast, in the rangewhere the brightness value is δ to β, there is a tendency of sharpdecrease in the number of pixels. Accordingly, the conversion curve inthe range where the brightness value is δ to β becomes a sharplydecreasing curve. The conversion curve that is illustrated in FIG. 6Bcan be obtained by joining these straight lines and curves together.

As described above, conversion curves can be obtained from thepost-clipping distribution maps FIG. 5A, FIG. 5B, FIG. 5D, and FIG. 5E.In the example FIG. 5A, FIG. 5B, and FIG. 5C where the set clippingvalue is large, α>α′ and β<β′, and Q<Q′ in the relation of the distanceQ between α to β and the distance Q′ between α′ to β′, in the conversioncurve. This indicates that the distribution of the pixels is expanded asα indicating the lower limit of the brightness value is decreased to α′and the β indicating the upper limit of the brightness value isincreased to β′. In other words, the levels of gradation are expandedafter the conversion compared with the levels of gradation of theoriginal image. Accordingly, portions with similar levels of brightnessvalue, which were hard to discriminate, can clearly be discriminated.

In the example FIG. 5D, FIG. 5E, and FIG. 5F where the set clippingvalue is small, in a similar manner to the example illustrated in FIG.5A, FIG. 5B, and FIG. 5C, α>α″ and β<β″, and Q<Q′ in the conversioncurve. However, the difference between these pairs of values are closeto each other. Accordingly, only a slight decrease and increase arepresent and there is not much difference after the conversion eventhough the lower limit decreases from α to α″ and the upper limitincreases from β to β″. In other words, the levels of gradation are notvery much changed after the conversion compared with the levels ofgradation of the original image. This indicates that almost no tonecorrection is done.

During or after the calculation of the conversion curve, α′ may beconverted such that α′=α, and β′ or the conversion curve may berecalculated accordingly. This is not limited to α′, but such conversionand recalculation may be performed for β′, α″, β″ in a similar manner tothe above.

According to the above description, it is desired that a fog image betreated as follows. An area with high optical transmittance is providedwith a smaller clipping value to decrease the degree of the contrastadjustment, and an area with low optical transmittance is provided witha larger clipping value to increase the degree of the contrastadjustment.

While the optical transmittance may be measured by the transmittancemeasuring sensor 17, the transmittance may be calculated from an imageby performing image processing. As a method of calculating thetransmittance, a method using a dark channel will be described below.For the detail of this method, see, for example, “Single image Hazeremoval Using Dark Channel Prior; He Kaiming, Jian Sun, Xiaoou Tang;Pattern Analysis and Machine Intelligence, IEEE 2011” andJP-2012-221237-A.

When the smallest RGB value at a local area of the fog image iscalculated from the properties of the fog, data that correlates with arough transmittance of the fog can be obtained. Such data is referred toas a dark channel. A model formula of the fog can be expressed in thefollowing formula 1, where J denotes the image data of cases where thereis no fog, t (t=0 to 1) denotes the transmittance of the air, A denotesthe light in the air (indirect-light components), and I denotes theimage data of cases where there is fog.I=J×t+(1−t)×A  [Formula 1]

In the formula 1 as above, “J×t” is referred to as direct-light data,and indicates the state in which an image when the fog with the opticaltransmittance of 100 percent clears off is attenuated by thetransmittance t. In the formula 1 as above, “(1−t)×A” is referred to asindirect-light data, and indicates the state in which the imageattenuated by the direct light is brightened by the light in the air A.

The dark channel holds for the property that the smallest RGB value ofmost substance is about zero when the fog clears off. According to theformula 1, as the transmittance t is smaller, the components of thelight in the air increase and the value of the light in the airincreases. On the other hand, the value of the direct-light data getsclose to zero as the transmittance t is smaller. Accordingly, thedirect-light data is assumed to be zero, and a rough estimate of thetransmittance t can be obtained from calculations using the value of thelight in the air A.

In order to make the minimum value of “J” even closer to zero, the localareas are set to be divided areas of small size such as “15×15”, and theminimum value can be calculated for each of these areas. It is to benoted that the smallest RGB value of a large white object or the likebecomes almost zero regardless of the state of the fog. For this reason,the transmittance of such a large white object or the like cannotprecisely be calculated using the dark channel.

Under the conditions as described above, the relational expressionbetween the dark channel and the transmittance t, which is derived fromthe model formula of the formula 1, is expressed in the followingformula 2. In the formula 2, “Ω” denotes the local area for a targetpixel, and “min_(c)” indicates the smallest RGB value.

$\begin{matrix}{t = {1 - {\min_{y \in \Omega}\left( {\min_{c}\left( \frac{J}{A} \right)} \right)}}} & \left\lbrack {{Formula}\mspace{14mu} 2} \right\rbrack\end{matrix}$

The formula 2 as described above is used to calculate and obtain thetransmittance t or the dark channel dark min_(yεΩ)(min_(c)(J)). In thefollowing description, the dark channel is referred to as dark(J). Thetransmittance calculator 22 illustrated in FIG. 2 uses the formula 2 asdescribed above to calculate and obtain the transmittance t or the darkchannel dark(J), and outputs the calculated transmittance astransmittance data to the contrast adjuster 23. The contrast adjuster 23determines the clipping value k to be used in the CLAHE according to thetransmittance t or the dark channel dark(J).

The clipping value k of the CLAHE may be determined without consideringthe above-described transmittance data, using the following formula 3and formula 4.

$\begin{matrix}{{k\_ min} = \frac{m}{N}} & \left\lbrack {{Formula}\mspace{14mu} 3} \right\rbrack\end{matrix}$k=k_min+(m−k_min)×S  [Formula 4]

In the formula 3, “m” denotes the number of the pixels of each tile, and“N” denotes the number of the bits of each pixel. The number of bits is256 for an 8-bit image, and 65536 for a 16-bit image. “k_min” denotes aminimum clipping value. In the formula 4, a set value S is a valueranging from 0 to 1. When the set value S is 1, it means a state wherethere is almost no constraint by the clipping value.

The clipping value of the CLAHE in view of the transmittance data may becalculated using the following formula 5 obtained by modifying theformula 4, where the average of the transmittance t per one tile is atile transmittance T, and the average of the dark channel dark(J) perone tile is a tile dark channel dark(J).k=k_min+(m−k_min)×S×{(1−T)×α1+β1}ork=k_min+(m−k_min)×S×{dark(J)×α2+β2}  [Formula 5]

By using the formula 5 as described above, the clipping value of theCLAHE varies for each transmittance of the tiles, and the clipping valueof a tile with low transmittance can be increased and the clipping valueof a tile with high transmittance can be decreased. In the formula 5,the values α1, β1, α2, and β2 indicate the degree of the weighting madeby the linear transformation of the transmittance t, and clipping isdone when the result of the linear transformation exceeds 1. Forexample, when the sensitivity of the transmittance is increased suchthat the clipping value largely varies even for a difference with smalltransmittance, the values of α1, α2, β1, and β2 may be determined suchthat the maximum value of the tile dark channel in the entire imagebecomes 1 and the minimum value of the tile dark channel becomes 0.

The contrast adjuster 23 uses the formula 5 as described above tocalculate the clipping value for each tile, and uses the clipping valueto performs the CLAHE processing. As described above, the formula 5 maybe used to calculate a clipping value. Alternatively, some clippingvalues that correspond to the transmittance data are set to a table orthe like, and a clipping value may be determined by referring to thetable. When the image is an RGB image, the RGB color space may beconverted into the YCbCr color space or the like before performing theCLAHE processing. In other words, the color space may be converted usingany known formula or the like. The contrast adjuster 23 may use theconverted Y (brightness value) to make the histogram flattened asdescribed above.

The contrast adjuster 23 may just perform the CLAHE processing, but thefog image may become achromatic and the color saturation may becomeinsufficient when only the CLAHE processing is done. In such cases, thecolor saturation may be enhanced by increasing the Cb components and theCr components in the YCbCr color space. Cb denotes the color differencecomponents indicating the hue and color saturation of blue color orbluish color, and Cr denotes the color difference components indicatingthe hue and color saturation of red color or reddish color.

The color saturation adjustment in which the gain of the Cb and Crcomponents is increased may be performed using the following formula 6.Note that in the formula 6, Cb and Cr takes a value varying between −1and 1. Moreover, in the formula 6, x of Cx indicates b or r. Cx′indicates the value of Cb or Cr where the color saturation has beenadjusted.0>CxCx′=−1+{(Cx+1)}^(1+[S×{(1−T)×α3+β3}])0≦CxCx′=1−{(1−Cx)}^(1+[S×{(1−T)×α3+β3}])or0>CxCx′=−1+{(Cx+1)}^[1+{S×(dark(J)×α4+β4}])0≦CxCx′=1−{(1−Cx)}^[1+{S×(dark(J)×α4+β4}])  [Formula 6]

In a similar manner to the clipping value of the CLAHE, in the formula 6as described above, the gain decreases as the transmittance is greater,and the gain increases as the transmittance is smaller. In a similarmanner to the clipping value of the CLAHE, in the formula 6 as describedabove, the values α3, α4, β3, and β4 indicate the degree of theweighting made by the linear transformation of the transmittance t.

FIG. 7A, FIG. 7B, and FIG. 7C are diagrams illustrating the gain used incolor saturation adjustment according to an embodiment of the presentinvention. Here, how the gain changes as the characteristic part of theformula 6 varies from 1 to 3 is described with reference to FIG. 7A,FIG. 7B, and FIG. 7C. FIG. 7A indicates the cases in which thecharacteristic part of the formula 6 is 1. FIG. 7B indicates the casesin which the characteristic part of the formula 6 is 2. FIG. 7Cindicates the cases in which the characteristic part of the formula 6 is3. When the characteristic part is 1, the ratio of Cx′ to Cx is 1. Asthe characteristic part increases to 2 and 3, the ratio increases, andthe gain also increases. The characteristic part of the formula 6includes the transmittance data. For this reason, the color saturationis adjusted using the formula 6 as described above, and the gain of thecolor saturation adjustment can be determined according to thetransmittance data.

FIG. 8 is a flowchart of a flow of the contrast adjustment performed bythe imaging device 10 provided with the image processing circuit 18illustrated in FIG. 2. The processes that are performed by the imagingdevice 10 are described below with reference to FIG. 8. The lens unit 11of the imaging device 10 is directed to an any desired object to becaptured, and the shutter button is depressed. After that, in the stepS810, the capturing unit 12 captures an image. In the presentembodiment, the fog is to be removed. However, as a matter of course,the bai, the PM 2.5, or the like other than the fog may be a target tobe removed. When the fog is to be removed, whether or not to adjust thefog may be determined manually through the operation unit 15. Theimaging device 10 removes the fog according to the determination madethrough the operation unit 15. Note that some type of the imaging device10 can automatically detect whether the fog is present in the capturedimage. Such a device can automatically adjust the fog without botheringthe user.

In the step S810, the controller 13 is notified of the depressed shutterbutton, and instructs the capturing unit 12 to start capturing an image.According to the instruction, the capturing unit 12 opens the shutter toexpose an imaging element, and photoelectrically converts the light thatenters the imaging element. As a result, image data is output. In thestep S820, the controller 13 receives the image data, and stores thereceived image data in the image memory 14.

In the step S830, the CTL I/F 20 of the image processing circuit 18obtains the image data from the image memory 14, and the ISP unit 21performs image processing such as shading correction. After that, theimage data is sent to the transmittance calculator 22 and the contrastadjuster 23. The transmittance calculator 22 uses the formula 2 asdescribed above to calculate and obtain transmittance data such as thetransmittance t or the dark channel dark(J). The transmittancecalculator 22 outputs the calculated transmittance data to the contrastadjuster 23.

In the step 840, the CLAHE processor 24 of the contrast adjuster 23 usesthe transmittance data to calculate and determine the clipping valueusing the formula 5 as described above. In the step S850, the CLAHEprocessor 24 uses the determined clipping value to flatten the histogramof each tile as described above. Accordingly, a conversion curve isobtained for each tile. Then, the CLAHE processor 24 uses the obtainedconversion curves to convert the brightness values of the pixels of eachtile. As described above, the CLAHE processor 24 adjusts the brightnesscomponents of the pixels. The CLAHE processor 24 sends the adjustedimage data to the color saturation adjuster 25.

In the step S860, the color saturation color saturation adjuster 25performs color saturation adjustment on the color difference componentsother than the brightness components in the pixels of the image data.The color saturation adjuster 25 uses the formula 6 as described aboveto increase the gain of the color saturation components. By so doing,the color saturation improves. When the color saturation of all thepixels have been adjusted, the contrast adjuster 23 assumes that thecontrast adjustment is complete, and stores the image data in the imagememory 14 through the CTL I/F 20. When the storage in the image memory14 is complete, the imaging device 10 terminates the process.

The imaging device 10 includes the lens unit 11. The lens unit 11includes a zoom lens that can capture an image upon changing the focallength and magnifying an object to be captured. The lens data that isset for the zoom lens includes setting information such as the angle ofview and the zoom magnification power. The angle of view or themagnifying power may be changed by modifying the setting information.The angle of view indicates the capturing range, and has a greatestvalue and a smallest value for a wide rim and a tele-rim, respectively.The wide rim indicates a state where the focal length of the zoom lensis made shortest, and the tele-rim indicates a state where the focallength of the zoom lens is made longest. The zoom magnification power isthe magnification power for the wide rim when the focal length of thezoom lens is changed from the wide rim. For example, when the focallength for the wide rim is 35 millimeters (mm) and the focal length atthat time is 70 mm, the zoom magnification power is 2. The lens datasuch as the angle of view or zoom magnification power of the lens unit11 is stored in the memory provided for the controller 13.

In the wide rim as illustrated in FIG. 9A, not only the person whosedistance to the imaging device 10 is short but also the mountain and skywhose distance to the imaging device 10 is long are captured.Accordingly, in the wide rim as illustrated in FIG. 9A, it is consideredthat the differences in transmittance on this captured single image arelarge. By contrast, in the tele-rim as illustrated in FIG. 9B, only thecar whose distance to the imaging device 10 is short is captured. Onthis captured single image, the differences in depth dimension aresmall. Accordingly, in the tele-rim as illustrated in FIG. 9B, it isconsidered that the differences in transmittance on the captured singleimage are small. In view of the above circumstances, in the method wherethe dark channel is used, it is desired that the correction for caseswith low accuracy of transmittance be performed. When such correction isperformed, the clipping value is widely changed for the wide rim, andthe clipping value is not so much changed for the tele-rim.

The angle of view or the zoom magnification power serves as a parameterindicating the degree to which a portion of the image is close to thewide rim or the tele-rim, and enables estimation of the degree ofvariations in the transmittance in the image. For this reason, the CLAHEprocessor 24 obtains the lens data from the controller 13 to determine amore appropriate clipping value using the obtained lens data and thedark channel.

The clipping value k can be calculated by the following formula 7obtained by modifying the formula 5 as described above, using lenssetting data L that corresponds to the lens data. The value of L is setto 1, 0, 0.5 for the wide rim, the tele-rim, and the middle pointbetween the wide rim and the tele-rim, respectively. Moreover, the valueof L becomes closer to 0 as it gets closer to the wide rim from themiddle point, and the value of L becomes closer to 1 as it gets closerto the tele-rim from the middle point. However, L is not limited to theabove configuration. For example, the focal length or angle of view ofthe lens may be converted into a numerical form, and the value of L maybe set to the obtained number.k=k_min+(m−k_min)×S×{1−L×(T×α1+β1)}ork=k_min+(m−k_min)×S×[1−L×{(1−dark(J)×α2++β2)}]  [Formula 7]

In the formula 7, α1, α2, β1, and β2 are the smallest and largest valuesof the transmittance of all the dark(J) or all the tiles, and α1, α2,β1, and β2 are normalized to between 0 and 1. The clipping value k isdetermined using the formula 7. When the value of L is large and thecloseness to the wide rim is indicated, the clipping value widelychanges. On the other hand, when the value of L is small and thecloseness to the tele-rim is indicated, the clipping value barelychanges.

As described above, the lens data may be used to improve the accuracy oftransmittance. Moreover, the lens data may be used not only to adjustthe brightness but also to adjust the color saturation, and the accuracyof the color saturation thereby improves. Accordingly, the values forthe color saturation adjustment can also be calculated using the lenssetting data L, and using the following formula 8 obtained by modifyingthe formula 6 as described above. In the formula 8, α3, α4, β3, and β4are normalized in a similar manner to α1, α2, β1, and β2 in the formula7 as described above.0>CxCx′=−1+{(Cx+1)}^(1+[S×{1−L×(T×α3+β3)}])0≦CxCx′=1−{(1−Cx)}^(1+[S×{1−L×(T×α3+β3)}])or0>CxCx′=−1+{(Cx+1)}^{1+(S×[1−L×{(1−dark(J))×α4+β4}])}0≦CxCx′=1−{(1−Cx)}^{1+S×[1−L×{(1−dark(J)×α4+β4)}])}  [Formula 8]

The flow of the processing in which lens data is obtained and a clippingvalue is determined and color saturation is adjusted according to theobtained lens data is briefly described with reference to FIG. 10. Asthe steps S1010 to S1030 are similar to the steps S810 to S830 depictedin FIG. 8, the description of these steps is omitted. In the step 1040,the CLAHE processor 24 of the contrast adjuster 23 obtains the lens datafrom the controller 13.

Although the processes in the steps S1050 to S1070 are similar to theprocesses in the steps S840 to S860 depicted in FIG. 8, the lens settinginformation L and the formulas 7 and 8 as described above are used todetermine a clipping value and adjust the color saturation in the stepsS1050 to S1070.

When the sky is captured in the fog image, the distance between theimaging device 10 and the sky is further than the distance between theimaging device 10 and a person or mountain, and thus the opticaltransmittance of the sky is small. For this reason, if the formula 5 asdescribed above is used to determine a clipping value, the clippingvalue becomes large. Moreover, if the CLAHE is performed, the contrastis greatly enhanced. When the fog comes up, the sky area does notinclude any particular information. In such cases, the enhancement ofcontrast just leads to an increase in noise. In order to avoid suchsituation, it is desired that the clipping value for the sky areadetected in an image be small regardless of the transmittance data. Byso doing, the noise can be reduced.

As illustrated in FIG. 11, the image processing circuit 18 may furtherbe provided with a sky-area detector 26 that detects the sky area of animage. The other aspects of the configuration are similar to theconfiguration illustrated in FIG. 2. The sky area can be detected usingany technology known in the art. For example, see “Single ImageAlgorithm Based on Sky Region Segmentation” (2013), InformationTechnology Journal 12(6): 1168-1175. In this known technology, noise isremoved using the morphology processing based on the characteristicsthat the sky area is wide, bright, and edgeless, and binarization isperformed. Alternatively, for example, a method in which machinelearning is used to recognize the sky area may be adopted.

The processes that are performed by the imaging device 10 provided withthe sky-area detector 26 are described below with reference to FIG. 12.The processes in the steps S1210 to S1230 are similar to the processesin the steps S810 to S830 depicted in FIG. 8 as well as the processes inthe steps S1010 to S1030 depicted in FIG. 10. Accordingly, thedescription of the processes in the steps S1210 to S1230 is omitted. Inthe step S1240, the sky-area detector 26 detects the sky area in theimage at the same time as when the transmittance calculator 22calculates and obtains the transmittance data. Alternatively, thesky-area detector 26 may detect the sky area in the image after thetransmittance calculator 22 has calculated and obtained thetransmittance data. Subsequently, the sky-area detector 26 sends thedata of the detected sky area to the CLAHE processor 24 of the contrastadjuster 23.

The processes in the steps S1250 to S1270 are similar to the processesin the steps S840 to S860 depicted in FIG. 8 as well as the processes inthe steps S1050 to S1070 depicted in FIG. 10. However, in thedetermination of the clipping value in the steps S1250, the detected skyarea serves, for example, to set the clipping value to a minimum value,and to inhibit the contrast adjustment. Note that the sky area maydirectly be output to the color saturation adjuster 25 just as it is. Insuch cases, the CLAHE processor 24 does not at all perform the CLAHEprocessing on the sky area.

As the contrast adjustment is not so much performed or not at allperformed on the sky area, the noise in the sky area is reduced, and theimage quality of the fog-adjusted image improves. In the presentembodiment, cases of the sky area are described as an example. In asimilar manner to this method, an area in which contrast adjustment isnot very much desired may be extracted, and the clipping value of theextracted area may be set to a minimum value. Moreover, the extractedarea may be output without performing the contrast adjustment thereon.

In the above description, an area in which the contrast adjustment isnot very much desired has been described. On the contrary, there existsan area in which the contrast adjustment is to be performed actively.For example, such an area includes a road area when an image is capturedby a vehicle-installed camera. When a vehicle-installed camera is usedas a part of a drive recorder, an objet to be captured is a vehicle orobject on the road. When a vehicle-installed camera is for automaticdriving, an objet to be captured includes a vehicle, a pedestrian, atraffic sign, and a traffic signal, which are mainly objects on theroad.

In such an area, a large value is set to the clipping value contrary tothe above-described area in which the contrast adjustment is not verymuch desired. Accordingly, the visibility of that area improves at alltimes regardless of the transmittance. In the present embodiment, casesin which such an area is a road area will be described as an example.However, no limitation is intended thereby. FIG. 13A is a diagramillustrating an example of the image captured by the imaging device 10,according to the present embodiment. FIG. 13B is a diagram illustratingthe image of FIG. 13A in which a road area is detected, according to thepresent embodiment.

Various kinds of technology to detect a road area have been known in theart. For example, see JP-07-028975-A. In the technology disclosed inJP-07-028975-A, the white lines on the right and left sides aredetected, and an area within these two white lines is recognized as alane. However, the road area is not just the lane on which the vehicletravels forward. It is desired that the area indicated by oblique linesas illustrated in FIG. 13B including the lane on which the vehicletravels from the opposite direction be detected.

In view of the above circumstances, as illustrated in FIG. 14, the imageprocessing circuit 18 may further be provided with a road-area detector27 that detects the road area. The other aspects of the configurationare similar to the configuration illustrated in FIG. 2. In a similarmanner to the sky-area detector 26, the road-area detector 27 obtainsthe image data from the ISP unit 21, and detects a road area from theimage of the obtained image data. The road-area detector 27 sends thedata of the detected road area to the contrast adjuster 23. The contrastadjuster 23 determines the clipping value such that the CLAHE processor24 improves the contrast of the road area.

In order to vary the clipping value depending on whether the areabelongs to a road area or whether the area is close to or far from theroad area, a for-road control parameter D is used. The formula 9 givenbelow indicates a formula to calculate the clipping value using such afor-road control parameter D. The for-road control parameter D provides“0” for a road area, and provides “1” for an area distant from the roadarea by the distance equal to or greater than a prescribed distance.Moreover, the for-road control parameter D provides a value ranging from0 to 1 for an area distant from the road area by the distance shorterthan the prescribed distance. Note that the value ranging from 0 to 1 ismade proportionate to the distance to the road area.k=k_min+(m−k_min)×S×{1−D×(T×α1+β1)}ork=k_min+(m−k_min)×S×[1−D×{(1−dark(J)×α2+β2)}]  [Formula 9]

By using the formula 9, the clipping value can be calculated in view ofthe transmittance data such that the clipping value becomes larger asthe area is closer to the road area and the clipping value becomessmaller as the area is further from the road area. In a similar mannerto the above, the for-road control parameter may be used not only toadjust the brightness but also to adjust the color saturation, and theaccuracy of the color saturation thereby improves. Its formula is givenas the formula 10 as below.0>CxCx′=−1+{(Cx+1)}^(1+[S×{1−D×(T×α3+β3)}])0≦CxCx′=1−{(1−Cx)}^(1+[S×{1−D×(T×α3+β3)}])or0>CxCx′=−1+{(Cx+1)}^{1+(S×[1−D×{(1−dark(J))×α4+β4}])}0≦CxCx′=1−{(1−Cx)}^{1+S×[1−D×{(1−dark(J)×α4+β4)}])}  [Formula 10]

The processes that are performed by the imaging device 10 provided withthe road-area detector 27 are described below with reference to FIG. 15.The processes in the steps S1510 to S1530 are similar to the processesin the steps S810 to S830 depicted in FIG. 8. Accordingly, thedescription of the processes in the steps S1500 to S1530 is omitted. Inthe step 1540, the road-area detector 27 detects a road area in theimage at the same time as when the transmittance calculator 22calculates and obtains the transmittance data. Alternatively, theroad-area detector 27 may detect a road area in the image after thetransmittance calculator 22 has calculated and obtained thetransmittance data. Subsequently, the road-area detector 27 sends thedata of the detected road area to the CLAHE processor 24 of the contrastadjuster 23.

The processes in the steps S1550 to S1570 are similar to the processesin the steps S840 to S860 depicted in FIG. 8. However, in thedetermination of the clipping value in the steps S1550, the road areaserves, for example, to set the clipping value of the road area to avalue greater than the clipping value of the other areas, and to enhancethe contrast adjustment.

In the above description, the processes in which the contrast of a fogimage is adjusted using the transmittance data have been described as anexample. When the contrast of a fog image is adjusted as describedabove, an image is divided into a plurality of rectangular tilesaccording to a division condition that an image is to be divided into aplurality of rectangular tiles, and a clipping value is determined foreach of the tiles. Then, the CLAHE is performed to adjust the colorsaturation. If the image is divided into tiles as above, like a house asillustrated in FIG. 16A, the parameters for the image processing of theobjects that extend over the boundary of tiles changes at theboundaries. For this reason, a difference will be present in the resultof the image processing. As a result, an image on which the imageprocessing has been performed becomes a somewhat unnatural image.

In order to avoid such situation, the technology is known in which theshapes of objects are extracted by performing texture analysis and thearea is divided for each of the objects, as disclosed in JP-3543103-B.By dividing the objects as described above, the above-mentionedunnatural image can be avoided. However, when the fog is thick in theimage as illustrated in FIG. 16B, it is difficult to extract the shapesof objects, and thus it is difficult to divide the area for each of theobjects. Even if the area is successfully divided, the length of timespent for such processing is enormous, and it is difficult to completethe processes in real time.

In order to improve the contrast affected by the fog, the area isdivided according to the degrees of influence brought by the fog, andthe contrast is adjusted for each of the divided areas. By so doing, anatural image where there are little blooming whites or blocked upshadows can be output. As a method of dividing the area, for example, amethod in which a dimmed image is used may be used. In such a method, adimmed image is generated by image processing, and the area is dividedinto a plurality of areas according to the pixel values of the generateddimmed image. As described above, when this method is used, the divisioncondition is a condition that an image is to be divided according to thepixel values of the generated dimmed image, in place of the conditionthat an image is to be divided into a specified number of tiles.

As will be described later in detail, a dimmed image in which the levelsof the pixel values are adjusted to five levels from white to black asillustrated in FIG. 17 is generated. Although cases in which the imageis divided into five levels are described herein, the image may bedivided into any number of levels. The original image is divided into aplurality of areas according to the pixel values of the dimmed image.The white area in the dimmed image is a sky area, and thus does notrequire any improvement in contrast. For this reason, as illustrated inFIG. 17, the image is divided into four areas. Note that the buildingand the street lamp in the image have the pixel values of the same levelin the dimmed image, and thus the building and the street lamp can betreated as a same region. Accordingly, the contrast can be adjusted foreach of the areas divided as above.

The dimmed image is described in detail with reference to FIG. 17. In adimmed image, the depth dimension is at a shorter distance as the pixelvalue is smaller (darker), and the depth dimension is at a longerdistance as the pixel value is larger (brighter). In the dimmed image,the levels of the depth dimension as described above are changedaccording to the distance from the imaging device 10 to an object to becaptured. Accordingly, the dimmed image can be generated based on thedistance data obtained by

distance measurement using image processing such as stereo matchingwhere a plurality of imaging device are used or

distance measurement using a laser radar or the like where the imageprocessing is not performed.

The brightness value of the dimmed image does not only depend on thedistance from the imaging device 10 to an object to be captured, butalso depends on the transmittance. The fog has a smaller transmittancewhen the fog is further from the imaging device 10, and has a largertransmittance when the fog is closer to the imaging device 10. In otherwords, the transmittance becomes larger when the pixel values of thedimmed image are small, and the transmittance becomes smaller when thepixel values of the dimmed image are large. For this reason, atransmittance measuring unit that measures the transmittance may be usedin place of the dimmed image, and the area may be divided according tothe transmittance measured by the measuring unit.

Next, a method of generating a dimmed image is described with referenceto (a) to (d) of FIG. 18. The method of generating a dimmed image issimilar to the above-described method using a dark channel. In thepresent example, the original image is an RGB image. The smallest RGBvalue is selected for each of the pixels that make up the original imageillustrated in (a) of FIG. 18. Each of the pixels has pixel values forthe respective R, G, and B, and the smallest value is selected fromthese pixel values. By selecting smallest RGB value as described above,an appropriate dimmed image can be generated for a light source such asthe street lamp, unlike the cases in which the level of lightness issimply selected.

Next, as illustrated in (b) of FIG. 18, the pixels in a prescribed area31 that includes a pixel (target pixel) 30 selected from the image forwhich smallest RGB values have been selected, excluding the target pixel30, are determined to be the peripheral pixels, and the pixel values ofthe target pixel 30 and the peripheral pixels are substituted by thesmallest RGB value of the peripheral pixels. By making full use of thedata of the peripheral pixels as described above, the pixel values ofobjects with the same distance to the imaging device 10 but with varyingpixel values, such as the wall and window of a building, can besubstituted by the same pixel value. The prescribed area 31 may bedetermined as desired. The prescribed area may be, for example, an areacomposed of nine pixels in total encompassing the target pixel 30 as thecenter.

If the pixel values are successively substituted according to the dataof the peripheral pixels, some pixel values that do not have to bereplaced may also be converted unnecessarily. For example, when only theperipheral pixels on the right and bottom sides of the target pixel 30are actually to be replaced, the peripheral pixels on the top and leftsides of the target pixel 30 may unnecessarily be replaced. In suchcases, the objects in the image expand as illustrated in (c) of FIG. 18.In (c) of FIG. 18, all the objects in the image, i.e., the mountains,the house, the street lamp, and the building, are expanded.

If the contrast is adjusted in such a state, a halo appears at theboundaries of the objects. In order to avoid the appearance of such ahalo, the edges are corrected such that the original shapes of theobjects match the shapes in the dimmed image. A general-purpose imageprocessing method such as a guided filtering where the original image isutilized may be used to correct the edges.

As described above, firstly, the smallest RGB values are selected, andsecondly, the pixel value of the target pixel 30 is replaced withsmallest RGB value of the peripheral pixels. Finally, the edges arecorrected, and the dimmed image can be generated as illustrated in (d)of FIG. 18. An RGB image having three factors is used in the presentembodiment, but no limitation is intended thereby. However, the methodas described above cannot be applied to color spaces such as Lab havinglevels of lightness and other factors and YUV having levels ofbrightness and other factors.

As illustrated in (a) to (d) of FIG. 18, a dimmed image may be generatedfrom an original image. Alternatively, as illustrated in FIG. 17, adimmed image may be generated using the distance data and thetransmittance data. Note that when the pixel values of an original imageare expressed by the 256-level gray scale, the pixel values of a dimmedimage are also expressed by the 256-level gray scale in the same way asthe original image. Accordingly, when an image is divided into areasbased on the pixel values, the pixel values are delimited into ranges asdesired and the pixels having the pixel values in the same range may bedefined to be in a same area.

FIG. 19A, FIG. 19B, and FIG. 19C are diagrams illustrating anothermethod of dividing the pixel values of a dimmed image, according to thepresent embodiment. FIG. 19A is a dimmed-image histogram with bar chartillustrating the relation between the pixel values of a dimmed image andthe number of pixels of the pixels having the corresponding pixel valueswhen the pixel values of the dimmed image ranges between 0 to 255. Inthe present embodiment, the area whose contrast is to be adjusted isdivided into four. For this reason, an example in which the dimmed imageis divided into four areas is described.

In order to divide the image into four areas, three thresholds are used.FIG. 19B illustrates a division method in which the variance is takeninto consideration. In this division method, the thresholds aredetermined such that the variance has the largest value at the largestand smallest thresholds. FIG. 19C illustrates a simple division methodas another alternative division method. In this division method, 256levels are equally divided into four ranges, and thus the thresholds aresimply at 64, 128, and 192. These methods are given as an example, andany other method may be adopted. Moreover, the number of partitions maybe other than four.

The dimmed image has been described so far in the above description. Inthe following description, the contrast adjustment to be performed foreach of the areas obtained by dividing an original image using a dimmedimage is described with reference to FIG. 20A, FIG. 20B, and FIG. 20C.FIG. 20A is an example of the histogram of an original image accordingto the present embodiment. FIG. 20B is an example of the cumulativehistogram obtained by adding and accumulating the number of the pixelsof each pixel value, according to the present embodiment. FIG. 20C is anexample of the conversion curve for flattening the histogram, accordingto the present embodiment.

When the original image is an image having three factors such as an RGBimage, the histogram is flattened for each of the three factors. Thehistogram may be flattened using the clipping value as described above.Firstly, a histogram as illustrated in FIG. 20A is calculated andobtained for each of the divided areas. Assuming that the pixel value isx, the histogram can be expressed as a function h(x). The number of thepixels of each pixel value is added and accumulated to obtain acumulative histogram. Assuming that the pixel value is x, the cumulativehistogram can be expressed as a function H(x). The function H(x) can beexpressed in the following formula 11. Note that when x=0, H(0)=h(0) inthe function H(x).H(x)=H(x−1)+h(x)  [Formula 11]

The clipping value is determined as described above, and the conversioncurve is calculated and obtained using the determined clipping value.Assuming that the input and output of the conversion curve is the pixelvalue x and the pixel value y, respectively, and that y=L(x), thefunction L(x) can be expressed in the following formula 12.

$\begin{matrix}{{L(x)} = \frac{256 \times {H(x)}}{H(255)}} & \left\lbrack {{Formula}\mspace{14mu} 12} \right\rbrack\end{matrix}$

As illustrated in FIG. 21, the image processing circuit 18 may furtherbe provided with a dimmed-image generator 28 that generates a dimmedimage. The other aspects of the configuration are similar to theconfiguration illustrated in FIG. 2. In a similar manner to the sky-areadetector 26 and the road-area detector 27, the dimmed-image generator 28obtains the image data from the ISP unit 21, and generates a dimmedimage from the image of the obtained image data. Alternatively, asdescribed above, a dimmed image may be generated using the measureddistance data or transmittance data. The dimmed-image generator 28 sendsthe data of the generated dimmed image to the contrast adjuster 23. Inthe contrast adjuster 23, the CLAHE processor 24 divides the receiveddimmed image into a plurality of areas according to the pixel values ofthe dimmed image, and determines the clipping value.

FIG. 22 is a flowchart of the flow of the contrast adjustment performedby the imaging device 10 provided with the image processing deviceillustrated in FIG. 21. The processes of dividing an image into aplurality of areas using the above-described dimmed image and adjustingthe contrast are described with reference to FIG. 22. The imaging device10 starts the processes when the shutter button is depressed. In thestep S2210, the capturing unit 12 captures an image. In the step S2220,the controller 13 stores the captured image in the image memory 14.

In the step S2230, the transmittance calculator 22 calculates andobtains the transmittance from the original image, or the transmittancemeasuring sensor 17 measures the transmittance. Accordingly, thetransmittance data of the area between the imaging elements and theobject to be captured is obtained. Moreover, the rangefinder measuresthe distance between the imaging device 10 and the objet to be captured,and thereby obtains the distance data. Note that the distance data maybe calculated and obtained by the image processing.

In the step S2240, the dimmed-image generator 28 generates a dimmedimage using the obtained transmittance data or distance data. Theoptical transmittance becomes smaller as the distance is longer, and theoptical transmittance becomes larger as the distance is shorter. Thedimmed-image generator 28 sets the pixel value to a smaller value as thetransmittance is larger or the distance is shorter, and sets the pixelvalue to a larger value as the transmittance is smaller or the distanceis longer. Accordingly, a dimmed image is generated. The dimmed-imagegenerator 28 sends the generated dimmed image to the CLAHE processor 24.

In the step S2250, the CLAHE processor 24 divides the received dimmedimage into a plurality of areas according to the pixel values of thedimmed image. Then, in the step 2260, the CLAHE processor 24 divides theoriginal image input from the ISP unit 21 into a plurality of areasaccording to the divided areas of the dimmed image. The dimmed image maybe divided using, for example, a dividing method where theabove-described variance is considered or a dividing method where thearea is equally divided. In the step S2270, the CLAHE processor 24determines the clipping value for each of the divided areas using themethod as described above.

In the step S2280, the CLAHE processor 24 uses the determined clippingvalue to flatten the histogram, and obtains a conversion curve. TheCLAHE processor 24 may obtain a LUT instead of the conversion curve. Theconversion curve or the LUT are calculated for each of the areas. Then,the CLAHE processor 24 uses the conversion curve or LUT calculated foreach of the areas to adjust the contrast of each one of the areas. Morespecifically, the CLAHE processor 24 uses the conversion curvecalculated for each of the areas to convert the pixel values of eacharea. The CLAHE processor 24 converts the pixel values of each area, andoutputs the obtained image.

In the step S2290, the color saturation adjuster 25 adjusts the colorsaturation of each area of the image, and outputs the contrast-adjustedimage. The contrast-adjusted image is sent to the image memory 14through the controller 13, and is stored until the contrast-adjustedimage is to be output to the output device. The process terminates, forexample, when the contrast-adjusted image is sent to the image memory 14and is stored.

As described above, the clipping value is a constraint value of thecontrast, and the size of the clipping value is changed according to thetransmittance of each area of the image. Accordingly, the contrast canbe adjusted appropriately, and the visibility of the image improves.Moreover, by dividing the area appropriately according to thetransmittance or distance instead of dividing the area into a pluralityof rectangular tiles, the visibility of the image further improves.

In the embodiments described above, an area whose histogram is to beflattened is segmented according to the transmittance, the distance, orthe like. Accordingly, the contrast of an image can be improved even ina thick-fog situation where it is difficult to detect a texture area.However, as the number of the partitions may be any desired number, aportion of the sky area or the like that is to be included in the samearea may be divided in an undesired manner. In such cases, a boundarymay be formed at a portion that is actually far from an appropriateboundary, and a false contour may appear. Moreover, as a result of thefog removal, the levels of lightness may be inverted from an image offine weather. In such cases, the improvement in the contrast of an imageis insufficient. In order to avoid such situation, a method of dividinga dimmed image, as described below, may be adopted to eliminate theoccurrence of a false contour, blooming whites, or blocked up shadows.By adopting such a method of dividing a dimmed image, the contrast of animage can further be improved.

FIG. 23 is a diagram illustrating another method of dividing the pixelvalues of a dimmed image, according to the present embodiment. Firstly,the division of the pixel values of a dimmed image is described withreference to FIG. 23. Assuming the histogram as illustrated in FIG. 23,the threshold is set to any desired value w, and the histogram isdivided into two at the threshold w. The image is classified into apattern region and a background region. The number of the pixels withpixel values less than the threshold is ω1, and the average and varianceare m1 and σ1, respectively. The number of the pixels with pixel valuesequal to or greater than the threshold is ω2, and the average andvariance are m2 and σ2, respectively. Accordingly, the interclassvariance σ_(b) ² can be expressed by the following formula 13.

$\begin{matrix}{\sigma_{b}^{2} = \frac{\omega\; 1\;\omega\; 2\left( {{m\; 1} - {m\; 2}} \right)^{2}}{\left( {{\omega\; 1} + {\omega\; 2}} \right)^{2}}} & \left\lbrack {{Formula}\mspace{14mu} 13} \right\rbrack\end{matrix}$

In the present embodiment, the value of “w” is varied using the formula13 as described above, and the threshold with which the interclassvariance σ_(b) ² becomes the largest is determined to be a firstthreshold. When the image data of an 8-bit image is used, the range ofvariation in “w” in the present embodiment is 0 to 255 according to thebit precision of the pixel value of the image data. When the image dataof a 10-bit image is used, the range of variation in “w” is 0 to 1023.In the mode of the sky area, the threshold of 0 to sky area is used.

Note that the formula that is used to determine the threshold w is notlimited to the formula 13 as described above, but may be any otherformula. The number of partitions may be any number. When the number ofpartitions is 2, only the threshold w needs to be determined.

FIG. 24 is a diagram illustrating another method of dividing the pixelvalues of a dimmed image into two or more ranges, according to thepresent embodiment. Secondly, the redivision of the pixel values of adimmed image is described with reference to FIG. 24. After the thresholdw is determined using the formula 13 as described above, the pixelvalues of the dimmed image equal to or smaller than the threshold w canfurther be divided into two regions. Assuming that the threshold of suchdivision is “u”, a threshold u is calculated using the formula 13 in asimilar manner to the threshold w. The threshold w is calculated usingthe same formula because the threshold is to be determined according tothe same criteria.

If the threshold with which the interclass variance σ_(b) ² becomes thelargest is determined to be a second threshold u in a similar manner tothe determination of the threshold w, the pixel values of the dimmedimage can be divided into three regions of 0 to u, u to w, and w to 255.In a similar manner, by calculating a threshold v for the range from thethreshold w to 255, the pixel values of the dimmed image can be dividedinto four ranges. The number of the regions is not limited to tworegions, three regions, and four regions, but the pixel values of thedimmed image may be divided into fiver or more regions.

By repeating the processes as described above, the region may be dividedinto optimal number of regions. Although the region may be divided intoany number of regions by setting a large number of thresholds, it is notdesired to divide the region when the interclass variance σ_(b) ² issmaller than a prescribed set value σ_(w). This is because a section tobe included in the same region is divided when an excessive number ofthresholds are set. In such cases, a false contour is generated asdescribed above.

When the interclass variance σ_(b) ² is always equal to or smaller thanthe set value σ_(w) regardless of the value of the threshold u in theexploration of the threshold u, no division is performed at thethreshold u and the process shifts to the exploration of the nextthreshold v. The set value σ_(w) may be selected as desired by a user,or may be determined based on the variance in the entire dimmed image.By repeating the processes as described above, the region can be dividedinto optimal number of regions, and the generation of a false contourcan be prevented. Note that the division and redivision as describedabove can be implemented by the contrast adjuster 23.

FIG. 25 is a flowchart of the processes of dividing the pixel values ofa dimmed image illustrated in FIG. 24. The division and redivision ofthe pixel values of a dimmed image is described with reference to FIG.25. In the step S2505, a dimmed image is obtained using the methoddescribed above with reference to FIG. 18. In the step S2510, ahistogram of the obtained dimmed image, as illustrated in FIG. 23, isgenerated according to the pixel values of the dimmed image that arecalculated for each of the pixels of the image data.

In the step S2515, the interclass variance σ_(b) ² is calculated usingthe formula 13 as described above in order to divide the region of thedimmed image using the generated histogram. In the step S2520, the σ_(b)² is calculated every time the threshold w for the area division isvaried to determine the maximum value Σ of σ_(b) ² and the threshold wfor the area division at the maximum value Σ is recorded.

In the step S2525, the maximum value Σ is compared with a prescribedthreshold σ_(w), and it is determined as to whether the maximum value Σis greater than the threshold σ_(w). When the maximum value Σ is equalto or smaller than the threshold σ_(w), the process proceeds to the stepS2555 and the process is terminated. In such cases, it is assumed thatno area division is to be performed, and the contrast is adjusted asillustrated in FIG. 20. When the maximum value Σ is greater than thethreshold σ_(w), the process proceeds to the step S2530, and thethreshold of area division w is determined to be a first threshold.

In the step S2535 and step S2540, as illustrated in FIG. 24, thethreshold u is calculated for further redividing the region divided inthe step S2530. The method of calculation is similar to that in the stepS2515 and step S2520. In the step S2545, in a similar manner to the stepS2525, the maximum value Σ is compared with a prescribed thresholdσ_(w), and it is determined as to whether the maximum value Σ is greaterthan the threshold σ_(w). When the maximum value Σ is equal to orsmaller than the threshold σ_(w), the process proceeds to the stepS2550. When the maximum value Σ is greater than the threshold σ_(w), theprocess returns to the step S2530 to determine the threshold of areadivision.

In the step S2550, whether there is any other area to be divided isdetermined. When there is no other area to be divided, the process isterminated. When there is another area to be divided, the processreturns to the step S2530 to determine the threshold of area divisionfor the remaining region. In the present embodiment, the remainingregion indicates the region with the pixel values equal to or greaterthan the threshold w. In the remaining region, the threshold v asdescribed above is determined. In the step S2550, whether or not toterminate the process is determined according to whether there is anyother area to be divided. However, whether or not to terminate theprocess may be determined according to whether the number of the dividedregions is equal to or greater than a certain number.

In the above description, the division and redivision of the pixelvalues of a dimmed image has been described. The entire processes ofadjusting the contrast, including the division and redivision of thepixel values of a dimmed image, are described with reference to FIG. 26.In the step S2610, the imaging device 10 obtains an original image. Notethat the original image is an RGB image.

In the step S2620, whether or not to calculate the transmittance betweenthe imaging device 10 and an objet to be captured or the distancebetween the imaging device 10 and the objet to be captured is determinedfor generating a dimmed image. When it is determined that thetransmittance or the distance be calculated by performing imageprocessing on the obtained original image, the transmittance or thedistance is calculated and the process proceeds to the step S2640. Whenit is determined that the transmittance or the distance be calculated bya transmittance measuring unit or rangefinder, the process proceeds tothe step S2630 and the transmittance or the distance is calculated.

In the step S2640, a dimmed image is generated from the transmittanceand distance obtained in the step S2620 or step S2630, using the methoddescribed above with reference to FIG. 18. In the step S2650, the dimmedimage is divided into multiple areas using the method described abovewith reference to FIG. 23 and FIG. 24. In the step S2660, the originalimage is divided into multiple areas using the method described abovewith reference to FIG. 25. In the step S2670, a contrast adjustmentparameter is generated using the method described above with referenceto FIG. 20.

In the step S2680, the contrast is adjusted using the contrastadjustment parameter generated for the original image in the step S2670,and the process is terminated. The contrast can be adjusted byflattening the histogram as illustrated in FIG. 20. Alternatively, thecontrast may be adjusted by adjusting the contrast linearly according tothe largest pixel value and the smallest pixel value in each of theareas. Moreover, the contrast may be adjusted by adopting, for example,a gamma correction in which the levels of gradation of an image isadjusted according to the curve of a desired gamma value such that thegamma indicating the response characteristic of the gradation of theimage will be 1.

In the above description, cases have been described in which an imagewhose contrast is to be adjusted is an image captured by the imagingdevice 10. However, the image whose contrast is to be adjusted is notlimited to an image captured by the imaging device 10. In other words,the image whose contrast is to be adjusted may be, for example, an imagereceived from the server through the network, and an image stored in arecording medium such as a compact disc read only memory (CD-ROM) and anSD card.

The image processing device and the image processing system are notlimited to the configurations illustrated in FIG. 2, FIG. 11, FIG. 14,and FIG. 21, and may include two of the sky-area detector 26, theroad-area detector 27, and the dimmed-image generator 28. Alternatively,the image processing device or the image processing system may includeall of the sky-area detector 26, the road-area detector 27, and thedimmed-image generator 28.

In the above description, the embodiments of the present invention havebeen described as the image processing device, the image processingsystem, the imaging device, the image processing method, and theprogram. However, no limitation is intended thereby. The embodiments ofthe present invention have been described above, but the presentinvention is not limited to those embodiments and various applicationsand modifications may be made without departing from the scope of theinvention. The embodiments of the present invention may include, forexample, a recording medium storing the program as described above, or aserver that provides the program through the network.

Numerous additional modifications and variations are possible in lightof the above teachings. It is therefore to be understood that within thescope of the appended claims, the disclosure of the present inventionmay be practiced otherwise than as specifically described herein. Forexample, elements and/or features of different illustrative embodimentsmay be combined with each other and/or substituted for each other withinthe scope of this disclosure and appended claims.

Further, as described above, any one of the above-described and othermethods of the present invention may be embodied in the form of acomputer program stored in any kind of storage medium. Examples ofstorage mediums include, but are not limited to, flexible disk, harddisk, optical discs, magneto-optical discs, magnetic tapes, nonvolatilememory cards, read only memory (ROM), etc. Alternatively, any one of theabove-described and other methods of the present invention may beimplemented by ASICs, prepared by interconnecting an appropriate networkof conventional component circuits, or by a combination thereof with oneor more conventional general-purpose microprocessors and/or signalprocessors programmed accordingly.

What is claimed is:
 1. An image processing system comprising: a dividerconfigured to divide an image into a plurality of areas according to apredetermined division condition; a measuring unit configured to measurean optical transmittance of each of the plurality of areas of the imageor data correlating with the optical transmittance as transmittancedata, or calculate the optical transmittance or the data correlatingwith the optical transmittance as the transmittance data; adetermination unit configured to determine a parameter for adjusting acontrast for each of the plurality of areas according to thetransmittance data measured or calculated by the measuring unit; and anadjuster configured to adjust a contrast of each one of the plurality ofareas using the parameter determined by the determination unit.
 2. Theimage processing system according to claim 1, wherein the measuring unitcalculates the transmittance data using a smallest pixel value of pixelsforming a local area of the image.
 3. The image processing systemaccording to claim 1, wherein the image is an image captured by animaging device including a lens configured to magnify and capture anobject to be captured, the image processing system further includes alens data acquisition unit configured to obtain lens data of the lens,and the determination unit determines the parameter according to thelens data and the transmittance data.
 4. The image processing systemaccording to claim 1, further comprising: a sky-area detector configuredto detect a sky area of the image, wherein the adjuster adjusts acontrast of the sky area using a predetermined parameter.
 5. The imageprocessing system according to claim 1, further comprising: a road-areadetector configured to detect a road area of the image, wherein theadjuster adjusts a contrast of the road area using a predeterminedparameter.
 6. The image processing system according to claim 1, furthercomprising: a dimmed-image generator configured to generate a dimmedimage using one of an optical transmittance calculated from the image,an optical transmittance measured by the measuring unit, a transmittancecalculated from a reflectance of an emitted laser beam, distance betweenan imaging device to an object to be captured measured by a rangefinder,and distance between a plurality of imaging devices and the object to becaptured calculated from parallaxes of a plurality of images captured bythe imaging devices, wherein the divider divides the image into theplurality of areas according to a pixel value of the dimmed image as thepredetermined division condition.
 7. The image processing systemaccording to claim 6, wherein the dimmed-image generator replaces apixel value of a pixel selected from a plurality of pixels of the imageand pixel values of peripheral pixels excluding the selected pixelwithin a prescribed area including the selected pixel with a smallestpixel value of the peripheral pixels, to generate the dimmed image. 8.The image processing system according to claim 6, wherein the dividerdetermines a specified pixel value of the dimmed image to be a thresholdas the predetermined division condition, and divides the image into theplurality of areas according to the determined threshold.
 9. The imageprocessing system according to claim 8, wherein the divider divides thepixel values of the dimmed image into two areas according to the pixelvalues of the dimmed image, calculates a variance between the two areas,and determines a pixel value of the variance with a largest value to bethe threshold.
 10. The image processing system according to claim 9,wherein the divider determines the pixel value of the variance with thelargest value to be the threshold when the pixel value of the variancewith the largest value is greater than a prescribed value.
 11. The imageprocessing system according to claim 10, wherein the divider furtherdivides the divided pixel values of the dimmed image into two areasaccording to the divided pixel values of the dimmed image, calculates avariance between the two areas, and determines two or more thresholdswhen the pixel value of the variance with the largest value is greaterthan a prescribed value.
 12. An imaging device comprising an capturingunit configured to capture an image; and the image processing systemaccording to claim
 1. 13. An image processing device comprising one ormore processors configured as: a divider configured to divide an imageinto a plurality of areas according to a predetermined divisioncondition; a measuring unit configured to measure an opticaltransmittance of each of the plurality of areas or data correlating withthe optical transmittance as transmittance data, or calculate theoptical transmittance or the data correlating with the opticaltransmittance from the image as the transmittance data; a determinationunit configured to determine a parameter for adjusting a contrast foreach of the plurality of areas according to the transmittance datameasured or calculated by the measuring unit; and an adjuster configuredto adjust a contrast of each one of the plurality of areas using theparameter determined by the determination unit.
 14. A method ofprocessing an image, the method comprising: dividing an image into aplurality of areas according to a predetermined division condition;measuring an optical transmittance of each of the plurality of dividedareas or data correlating with the optical transmittance astransmittance data; calculating the optical transmittance or the datacorrelating with the optical transmittance from the image as thetransmittance data; determining a parameter for adjusting a contrast foreach of the plurality of areas according to the transmittance dataobtained by the measuring or the calculating; and adjusting a contrastof each one of the plurality of areas using the parameter determined bythe determining.
 15. A computer-readable non-transitory recording mediumstoring a program for causing a computer to execute the method accordingto claim 14.